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Abstract 

We introduce several generalizations of classical computer science problems obtained by 
replacing simpler objective functions with general submodular functions. The new problems 
include submodular load balancing, which generalizes load balancing or minimum-makcspan 
scheduling, submodular sparsest cut and submodular balanced cut, which generalize their re- 
spective graph cut problems, as well as submodular function minimization with a cardinality 
lower bound. We establish upper and lower bounds for the approximability of these problems 
with a polynomial number of queries to a function- value oracle. The approximation guarantees 
for most of our algorithms are of the order of -^/n/lnn. We show that this is the inherent 
difficulty of the problems by proving matching lower bounds. 

We also give an improved lower bound for the problem of approximating a monotone submod- 
ular function everywhere. In addition, we present an algorithm for approximating submodular 
functions with special structure, whose guarantee is close to the lower bound. Although quite 
restrictive, the class of functions with this structure includes the ones that are used for lower 
bounds both by us and in previous work. This demonstrates that if there are significantly 
stronger lower bounds for this problem, they rely on more general submodular functions. 

1 Introduction 

A function / defined on subsets of a ground set V is called submodular if for all subsets S", T C y, 
f{S) + f{T) > f{S U T) + f{S n T). Submodularity is a discrete analog of convexity. It also shares 
some nice properties with concave functions, as it captures decreasing marginal returns. Submod- 
ular functions generalize cut functions of graphs and rank functions of matrices and matroids, and 
arise in a variety of applications including facility location, assignment, scheduling, and network 
design. 

In this paper, we introduce and study several generalizations of classical computer science 
problems. These new problems have a general submodular function in their objectives, in place 
of much simpler functions in the objectives of their classical counterparts. The problems include 
submodular load balancing, which generalizes load balancing or minimum-makespan scheduling, and 
submodular minimization with cardinality lower bound, which generalizes the minimum knapsack 

*This work supported in part by NSF grant CCF-0728869. A preliminary version of this paper has appeared in 
the Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science. 
^Department of Computing Science, University of Alberta, Canada. 
"^Department of Computer Science, Dartmouth, USA. 
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problem. In these two problems, the size of a collection of items, instead of being just a sum of their 
individual sizes, is now a submodular function. Two other new problems are submodular sparsest 
cut and submodular balanced cut, which generalize their respective graph cut problems. Here, a 
general submodular function replaces the graph cut function, which itself is a well-known special 
case of a submodular function. The last problem that we study is approximating a submodular 
function everywhere. All of these problems are defined on a set V oi n elements with a nonnegative 
submodular function / : 2^ — )• ]R>o. Since the amount of information necessary to convey a general 
submodular function may be exponential in n, we rely on value-oracle access to / to develop 
algorithms with running time polynomial in n. A value oracle for / is a black box that, given a 
subset S, returns the value f{S). The following are formal definitions of the problems. 

Submodular Sparsest Cut (SSC): Given a set of unordered pairs {{ui,Vi} \ Ui,Vi € V}, each 
with a demand di > 0, find a subset 5 C F minimizing fiS)/'^-.^gi-^^^, Vi}\=i '^^^ denominator 
is the amount of demand separated by the "cut" (S", In uniform SSC, all pairs of nodes have 
demand equal to one, so the objective function is Another special case is the weighted 

SSC problem, in which each element v £ V has a non-negative weight w{v), and the demand 
between any pair of elements {u,v} is equal to the product w{u) ■ w{v). 

Submodular 6-Balanced Cut (SBC): Given a weight function w : V ^ ^>o, a cut {S, S) is 
called 6-balanced (for 6 < ^) if w{S) > b • w{V) and w{S) > b ■ w{V), where w{S) = X]i.e5^(^)- 
The goal of the problem is to find a 6-balanced cut {S, S) that minimizes f{S). In the unweighted 
special case, the weights of all elements are equal to one. 

Submodular Minimization with Cardinality Lower Bound (SML): For a given W >0, 
find a subset 5 C 1/ with |5| > W that minimizes f{S). A generalization with 0-1 weights 
w : V ^ {0, 1} is to find 5 with w{S) > W minimizing f{S). 

Submodular Load Balancing (SLB): The uniform version is to find, given a monoton^ 
submodular function / and a positive integer m, a partition of V into m sets, Vi, . . . , Vm (some 
possibly empty), so as to minimize maxj f{Vi). The non-uniform version is to find, for m monotone 
submodular functions fi, ■ ■ ■ , fm on V, a partition Vi, . . . , Vm that minimizes maxj fi{Vi). 

Approximating a Submodular Function Everywhere: Produce a function / (not nec- 
essarily submodular) that for all 5 C y satisfies f{S) < f{S) < 7(n)/(5), with approximation 
ratio 7(n) > 1 as small as possible. We also consider the special case of monotone two-partition 
functions, which we define as follows. A submodular function / on a ground set F is a two-partition 
(2P) function if there is a set R CV such that for all sets S, the value of f{S) depends only on the 
sizes ISTli^l and |5n^|. 

1.1 Motivation 

Submodular functions arise in a variety of contexts, often in optimization settings. The prob- 
lems that we define in this paper use submodular functions to generalize some of the best-studied 
problems in computer science. These generalizations capture many variants of their corresponding 
classical problems. For example, the submodular sparsest and balanced cut problems generalize 
not only graph cuts, but also hypergraph cuts. In addition, they may be useful as subroutines for 
solving other problems, in the same way that sparsest and balanced cuts are used for approximating 
graph problems, such as the minimum cut linear arrangement, often as part of divide-and-conquer 

^For any set S C V, we use S to denote its complement set, V \S. 
^ A function / is monotone if < f(T) whenever 5* C T. 
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schemes. The SML problem can model a scenario in which costs follow economies of scale, and 
a certain number of items has to be bought at the minimum total cost. An example application 
of SLB is compressing and storing files on multiple hard drives or servers in a load-balanced way. 
Here the size of a compressed collection of files may be much smaller than the sum of individual 
file sizes, and modeling it by a monotone submodular function is reasonable considering that the 
entropy function is known to be monotone and submodular jlOj . 

1.2 Related work 

Because of the relation of submodularity to cut functions and matroid rank functions, and their ex- 
hibition of decreasing marginal returns, there has been substantial interest in optimization problems 
involving submodular functions. Finding the set that has the minimum function value is a well- 
studied problem that was first shown to be polynomially solvable using the ellipsoid method |15U16j . 
Further research has yielded several more combinatorial approaches [9l l20f[22t l24 y32ll33ll35] . 

Submodular functions arise in facility location and assignment problems, and this has spawned 
interest in the problem of finding the set with the maximum function value. Since this is NP- 
hard, research has focused on approximation algorithms for maximizing monotone or non- monotone 
submodular functions, perhaps subject to cardinality or other constraints [3t[8| [25H271 [3 H [36]. A 
general approach for deriving inapproximability results for such maximization problems is presented 
in 00]. 

Research on other optimization problems that involve submodular functions includes [H [511181 
[38 l l39 1 l4Tj. Zhao et al. [42] study a submodular multiway partition problem, which is similar to our 
SLB problem, except that the subsets are required to be non-empty and the objective is the sum of 
function values on the subsets, as opposed to the maximum. Subsequent to the publication of the 
preliminary version of this paper, generalizations of other combinatorial problems to submodular 
costs have been defined, with upper and lower bounds derived for them. These include the set 
cover problem and its special cases vertex cover and edge cover, studied in [23], as well as vertex 
cover, shortest path, perfect matching, and spanning tree studied in [12]. In [12], extensions to the 
case of multiple agents (with different cost functions) are also considered. 

Since it is impossible to learn a general submodular function exactly without looking at the func- 
tion value on all (exponentially many) subsets [3, there has been recent interest in approximating 
submodular functions everywhere with a polynomial number of value oracle queries. Goemans et 
al- [H] give an algorithm that approximates an arbitrary monotone submodular function to a factor 
7(77.) = 0{^/nlogn), and approximates a rank function of a matroid to a factor 7(71) = \/n + 1. A 

lower bound of 0, ^^^^ for this problem on monotone functions and an improved lower bound of 

O (y^i^) for non-monotone functions were obtained in [131114]. These lower bounds apply to all 
algorithms that make a polynomial number of value-oracle queries. 

All of the optimization problems that we consider in this paper are known to be NP-hard 
even when the objective function can be expressed compactly as a linear or graph-cut function. 
While there is an FPTAS for the minimum knapsack problem [llj . the best approximation for load 
balancing on uniform machines is a PTAS |19] . and on unrelated machines the best possible upper 
and lower bounds are constants [29]. The best approximation known for the sparsest cut problem 
is 0{\/logn) [Hll], and the balanced cut problem is approximable to a factor of O(logn) |34j . For 
the special case of SML on graphs, introduced in [53 , an O(logn) approximation is possible using 
the recent results of Racke [3l] . 
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1.3 Our results and techniques 

We establish upper and lower bounds for the approximability of the problems listed above. Surpris- 
ingly, these factors are quite high. Whereas the corresponding classical problems are approximable 
to constant or logarithmic factors, the guarantees that we prove for most of our algorithms are of the 
order of y^j^- We show that this is the inherent difficulty of these problems by proving matching 
(or, in some cases, almost matching) lower bounds. Our lower bounds are unconditional, and rely 
on the difficulty of distinguishing different submodular functions by performing only a polynomial 
number of queries in the oracle model. The proofs are based on the techniques in [51[T3]. To prove 
the upper bounds, we present randomized approximation algorithms which use their randomness 
for sampling subsets of the ground set of elements. We show that with relatively high probability 
(inverse polynomial), a sample can be obtained such that its overlap with the optimal set is sig- 
nificantly higher than expected. Using the samples, the algorithms employ submodular function 
minimization to find candidate solutions. This is done in such a way that if the sample does indeed 
have a large overlap with the optimal set, then the solution satisfies the algorithm's guarantee. 

For SSC and uniform SLB, we show that they can be approximated to a factor. For 

SBC, we use the weighted SSC as a subroutine, which allows us to obtain a bicriteria approximation 
in a similar way as Leighton and Rao |28j do for graphs. For SML, we also consider bicriteria results. 
For p > 1 and < a < 1, a (p, (T)-approximation for SML is an algorithm that outputs a set S such 
that f{S) < pB and w{S) > aW, whenever the input instance contains a set U with f{U) < B 
and 'w{U) > W. We present a lower bound showing that there is no (p, c) approximation for any p 
and a with f = o {\/\^) ■ For 0-1 weights, we obtain a (Sy^j^, approximation. This algorithm 
can be used to obtain an 0{Vn\nn) approximation for non-uniform SLB. 

We briefly note here that one can consider the problem of minimizing a submodular function 
with an upper bound on cardinality (i.e., minimize f{S) subject to \S\ < W). For this problem, 
a (^, bicriteria approximation is possible for any < a < 1, using techniques in [l7j. For 
non-bicriteria algorithms, a hardness result of Q (y^i^) follows by reduction from SML, using the 
submodular function /, defined as f{S) = f{S), and a cardinality bound W = n — W . 

For approximating monotone submodular functions everywhere, our lower bound is (\/bin) > 
which improves the bound for monotone functions in [13^1141 . and matches the lower bound for 
arbitrary submodular functions, also in |13pi4j. Our lower bound proof for this problem, as well 
as the earlier ones, use 2P functions, and thus still hold for this special case. We show that 
monotone 2P functions can be approximated within a factor 0{\/n). Besides leaving a relatively 
small gap between the upper and lower bounds, this shows that if much stronger lower bounds for 
the approximation problem exist, they rely on more general submodular functions. 

For the problems studied in this paper, our lower bounds show the impossibility of constant or 
even polylogarithmic approximations in the value oracle model. This means that in order to obtain 
better results for specific applications, one has to resort to more restricted models, avoiding the 
full generality of arbitrary submodular functions. 

2 Preliminaries 

In the analysis of our algorithms, we repeatedly use the facts that the sum of submodular functions 
is submodular, and that submodular functions can be minimized in polynomial time. For example, 
this allows us to minimize (over T C.V) expressions like /(T) — a • |r n 5|, where a is a constant 
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and S is a fixed subset of V. 

We present our algorithms by providing a randomized relaxed decision procedure for each of the 
problems. Given an instance of a minimization problem, a target value B, and a probability p, 
this procedure either declares that the problem is infeasible (outputs fail), or finds a solution to 
the instance with objective value at most ^B, where 7 is the approximation factor. We say that 
an instance is feasible if it has a solution with cost strictly less than B (we use strict inequality 
for technical reasons; this can be avoided by adding a small value e > to -B). The guarantee 
provided with each decision procedure is that for any feasible instance, it outputs a 7-approximate 
solution with probability at least p. On an infeasible instance, either of the two outcomes is allowed. 
Randomized relaxed decision procedures can be turned into randomized approximation algorithms 
by finding upper and lower bounds for the optimum and performing binary search. Our algorithms 
run in time polynomial in n and In 

Let us say that an algorithm distinguishes two functions /i and /2 if it produces different output 
if given (an oracle for) /i as input than if given (an oracle for) f2- The following result is used for 
obtaining all of our lower bounds. 

Lemma 2.1 Let fi and /2 be two set functions, with f2, hut not fi, parametrized by a string 
of random bits r. If for any set S, chosen without knowledge of r, the probability (over r) that 
fi{S) 7^ f2{S) is n~'^^^\ then any algorithm that makes a polynomial number of oracle queries has 
probability at most n~^^^^ of distinguishing fi and f2- 

Proof. We use reasoning similar to [8j. Consider first a deterministic algorithm and the compu- 
tation path that it follows if it receives the values of fi as answers to all its oracle queries. Note 
that this is a single computation path that does not depend on r, because /i does not depend on 
r. On this path the algorithm makes some polynomial number of oracle queries, say n'^. Using the 
union bound, we know that the probability that /i and /2 differ on any of these n'^ sets is at most 

• n^'^^^^ = n~'^^^\ So, with probability at least 1 — n^'^^^\ if given either fi or /2 as input, the 
algorithm only queries sets for which /i = /2, and therefore stays on the same computation path, 
producing the same answer in both cases. 

A randomized algorithm can be viewed as a distribution over a set of deterministic algorithms. 
Since, by the discussion above, each of these deterministic algorithms has probability at most n~^^^^ 
of distinguishing fi and /2, the randomized algorithm as a whole also has probability at most n^^^^^ 
of distinguishing these two functions. □ 

The following theorem about random sampling is used for bounding probabilities in the analyses 
of our algorithms. We use the constant c = l/(4\/27r) throughout the paper. 

Theorem 2.2 Suppose that m elements are selected independently, with probability < q < 1 
each. Then for < e < the probability that exactly \qm{l + e)] elements are selected is at 

least cq ■ m 2 ■ exp 
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Proof. Let A = qm{l + e). First we consider the case that A is integer. For convenience, let 
K = q{l + e), and note that k < 1. Using an approximation that -v/2vrn (f )" < n\ < l^phrn (f 
which is derived from Stirling's formula P, p. 55], we obtain the bound 



m\ ml ^ VZTT i/m {m/eY 



{mK)\{m — mK)l (2\/27r)2 yJmK\Jm — mn {mn / e)^'^ {{m — mK) / e) 



5 



> 



1 



1 



1 



Let X be the number of elements selected in the random experiment. Then 
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where we have used the inequality that 1 + x < for all x. The assumption that e < ensures 
that the denominator 1 — q{l + e) is positive. Now, the exponent of e is equal to 



-eqm{l + e) + 



eq 



1 



■m(l 



eqm — e qm + eqm 



2 2 

eqm 
1-0 



-e^qm 



Noting that c - m 2 > cq ■ m 2 concludes the proof for the case that A is integer. 
If A is fractional, then [A] = [AJ +1. Then 



(m- LAJ) 



Pr[X= LAJ] 



(-) gLAJ (l-5)--LAJ 



(LAJ+1) {l-qY 



(1) 



As e > 0, we have A > qm. Now consider the case that [AJ < qm. As qm is the expectation of X, 
either [A] or [AJ is the most likely value of X, having probability of at least In the first case, 

Pr[X = [AJ] > > ^, and we are done. In the second case, using sequentially ([T|), [AJ < qm, 
and [AJ + 1 = [AJ < m (which is implied by k < 1 above), we obtain the result: 



Pr[X= [AJJ > 



(m-[AJ) 



> 



mq 



m + 1 ([AJ+l)(l-g) - m + 1 [AJ + 1 



> ^ 
~ m 



The remaining case is that [AJ > qm. Define e' > to be such that qm{l + e') = [qm{l + e)\ 
[AJ . Note that e' < e. Applying the proof that we used for integer A, we obtain that 



Pr[X= [AJ] > 



m 



■ exp 



12 

-e qm 



> 



m 



■ exp 



-e^qm 



where we also used monotonicity of the exponential function. Using the fact that [AJ < m — 1, 
we simplify equation ([T|) to obtain that Pic[X = [AJ]/Pr[X = [AJ] > ^. Together with the above 
inequality, this gives the desired result. □ 
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3 Submodular sparsest cut and submodular balanced cut 



3.1 Lower bounds 

Let e > be such that = ^ -(^(Inn), let /3 = j{l + and let i? be a subset of V of size ^, with 
parameters such that n is even and /3 is an integer. We define the following two functions, and 
show that they are submodular and hard to distinguish. Moreover, these functions are symmetric!!. 

MS) = min(|5|, ^)-^ 

/2(5) = min(|5|, ^, /3 + |Sni?|, /3 + |5ni?|) 

Lemma 3.1 Functions fi and f2 defined above are nonnegative, submodular, and symmetric. 

Proof. The first function can be written as fi{S) = ^minds'], 151), which makes it easy to see 
that it is nonnegative and symmetric. It suffices to show that f{S) = mindSI, ^) is submodular, 

since — is modulaiQ. We use an alternative definition of submodularity: / is submodular if for 
all 5 C y and a,heV\S, with a / 6, it holds that /(5U {a, b}) - f{SU{b}) < /(5U {a}) - f{S). 
The only way that this inequality can be violated for our function is if f{SU{a, b}) — f{SU{b}) = 1 
and f{SL){a}) — f{S) = 0. But this is a contradiction, since the second part implies that |5| > n/2, 
and the first one implies that ISUI^}! < n/2. 

To see that /a (5) is nonnegative, we note that /3+|5ni?|-^>f + |5ni?|- - > 0, 

since IS" n i?| < ^. A similar calculation shows that (3 + \S H R\ — >0, and thus f2{S) > for 
all S. To show symmetry, we use the fact that |i2| = ^, and thus 

|5nii|-^ = --|5ni?|-^ = ^-|5ni?| = -^ + |5|-|5nii| = |5ni2|-^. 
' '22' '22' ' 2 2 

Analogously, \S Ci R\ - = \S D R\ - Thus, we have that 

f2{S) = min(M, M, ^ + |5ni?|-^, /3 + |5ni?|-M 

= minfM, ^ + |5ni?|-^, /3 + |5ni?|-^) =h{S). 



For submodularity of /a, we focus only on f{S) = min (|5|, ^,/3+ |5n R\,P + \S H R\). Sup- 
pose for the sake of contradiction that for some a,b & V, we have f{S U {a, b}) — f{S U {b}) = 1 
but f{S U {a}) — f{S) = 0. We assume that a £ R (the case that a G is similar). First consider 
the case that b is also in the set R. In this case f{S U {a}) = f{S U {b}). The fact that the 
function value does not increase when a £ R is added to S means that the minimum is achieved 
by one of the terms that do not depend on 15 H namely f{S) = min(^, /3 + H fi|). But then 
the minimum would also not increase when the second element of R is added, and we would have 
f{S U {a, b}) = f{S U {b}), contradicting the assumption. 

The remaining case is that a £ R and b £ R. As before, f{S) = min(|^,/3 + |5 n R\). But if 
f{S) = ^, then f{SU{a,b}) = ^, which contradicts our assumptions. So f{S) = /3+ jS'nfil. Now, 

''a function / is symmetric if f{S) = f{S) for all S. 

modular function is one for which the submodular inequality is satisfied with equality. 
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f{S U {b}) increases from the addition of a G i?, which means that its minimum is achieved by a 
term that depends on |5n f{SU{b}) = min(|5| + !,/? + |5n Suppose that f{SU{b}) = 
\S\ + 1. This means that \S\ + I < [3 + \ {S U {b}) n R\ = (3 + \S n R\ + I. But we also know 
that P + \SnR\ < \S\ jfrom the fact that f{S) = ^ + \SnR\). Thus, |5| = p + \SnR\ and 
f{S U {b}) = /3 + \S n R\ + I = (3 + \ {S U {b}) n R\. But this term does not depend on \S n R\, 
so adding a £ R to SU {b} would not change the function value, a contradiction. Finally, suppose 
that f{S U {b}) = P + \SnR\. As f{S) = ^ + \S n R\, we know that ^ + \S n R\ < \S\, and 
therefore l3 <\SnR\. So f{SLI {b}) = /3 + |5 n i?| > 2/3 > §, by the definition of /3. But this is a 
contradiction, as the value of / is always at most ^. □ 

To give a lower bound for SSC and SBC, we prove the following result and then apply Lemma 
l2.1l to show that the functions /i and /2 above are hard to distinguish. 

Lemma 3.2 Fix an arbitrary subset S V, and then let R be a random subset of V of size ^. 
Then the probability (over the choice of R) that fi{S) ^ f2{S) is at most n^'^^^\ 

Proof. We note that fi{S) / f2{S) if and only if min(/3 +_\S n R\, /3 + \S n R\) < min(|S|, f ). 
This happens if either 13 + \S n R\ < mindS"], §) or [3 + \S n R\ < mindS], §). The probabilities of 
these two events are equal, so let us denote one of them by p{S). If we show that p{S) = n"'^^-'^), 
then the lemma follows by an application of the union bound. 

First, we claim that p{S) is maximized when \S\ = ^. For this, suppose that \S\ > ^. Then 
p{S) = Pr[/3 + n i2| < But this probability can only increase if an element is removed from 
S. Similarly, in the case that |5| < f , p{S) = Pr[/3 + |5 n i?J < l^]] = Pr[/3 < \S D R\]. But this 
probability can only increase if an element is added to S. 

For a set S of size §, p{S) = Pr[/3 + |5 n i?| < f ] = Pr[|5 n R\ < ^(1 - e)]. If instead of 
choosing i? as a random subset of V of size ^ , we consider a set R' for which each element is chosen 
independently with probability |, then p{S) becomes 



piS) 



Pr 



n 



ISnR'l < -a-e) 



\R'\ 



Pr[|5ni?'| < f(l-e) A|i?'| 



n 
2 
f] 



Pr lii' 



< (n + l)-Pr ysni^'l < -(l-e) 



This allows us to make a switch to independent variables, so that we can use Chernoff bounds |30] . 
The expectation /_f of |5 n R'\ is equal to \S\/2 = n/A, so 



Pr [\Sr\R!\ < {l-e)fi] < e 



-Lj(ln n) 



remembering that = - • a;(lnn). This gives p{S) < (n + 1) • n ^^^^ 



n 



n 



-<^(i) 



□ 



Corollary 3.3 Any algorithm that makes a polynomial number of oracle queries has probability at 
most n""^^^^ of distinguishing the functions fi and f2- 

We now use these results to establish the hardness of the SSC and SBC problems. For concrete- 
ness, assume that the output of an approximation algorithm for one of these problems consists of 
a set 5 C 1/ as well as the value of the objective function on this set. 
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Theorem 3.4 The uniform SSC and the unweighted SBC problems (with balance b = 0(1) ) cannot 
be approximated to a ratio a ( y^j^) in the oracle model with polynomial number of queries, even 
in the case of symmetric functions. 

Proof. Suppose for the sake of contradiction that there is a polynomial-time 7-approximation 
algorithm for the uniform SSC problem, for some 7 = (a/j^) , that succeeds with high probability. 
We set e = 2^ with some 6 > I such that /3 = j{l + e) is integer. This satisfies = :^ • 

a;(lnn). One feasible solution for the uniform SSC on /2 is the set R, with ratio ^^^j^ = So 
if the algorithm is given function f^ as input, then with high probability it has to output a set 
5 with ratio /2(5)/|5||5'| < ^ = < However, for the function /i, the ratio of any set is 
1/2 max (I SI, l^l) > So if the algorithm is given /i as input, its output value differs from the 
case of /2 . But this contradicts Corollary 13.31 

For the lower bound to the submodular balanced cut problem, we consider the same two func- 
tions fi and /2 and unit weights. Assuming that there is a 7-approximation algorithm for SBC, we 
set e = 1^, with 7 > 1 ensuring the integrality of /3. This satisfies = ;^ • a;(lnn) if 7 = o (y^j^) 
and 6 is a constant. Since one feasible 6-balanced cut on /2 is the set R, whose function value is 
the algorithm outputs a 6-balanced set S with f2{S) < ^ne/A = bn/25 < bn/2. However, for any 
6, the optimal 6-balanced cut on /i is a set of size 6n, whose function value is bn/2. Thus, given 
/i, the algorithm would produce a different output, leading to a contradiction. □ 



3.2 Algorithm for submodular sparsest cut 

Our algorithm for SSC uses a random set S to assign weights to nodes (see Algorithm [T]) . For 
each demand pair separated by the set 5", we add a positive weight equal to its demand di to the 
node that is in S, and a negative weight of —di to the node that is outside of S. This biases the 
subsequent function minimization to separate the demand pairs that are on different sides of S. 



Algorithm 1 Submodular sparsest cut. Input: V , f, d, B, p 



for ^ln(Y^) iterations do 

Choose a random set S by including each node v ^ V independently with probability ^ 
for each v £ V, initialize a weight w{v) = 
for each pair {ui,Vi} with \{ui,Vi} H = 1 do 

Let Si £ {ui,Vi} n S and ti G {uj, Vi} \ S > name the unique node in each set 

Update weights w{si) ^ w{si) + d^; w{ti) ^ w{ti) — di 
end for 

Let T be a subset of V minimizing /(T) — a ■ X]»;eT w{v) 

if f{T) — a-Yl,v^T'^i'^)'^^^ return T 
end for 
return fail 



Lemma 3.5 If for some set T '^V , it holds that f{T) — a ■ ^^^j^ w{v) < 0, then 

< a. 

Si:|TnK,i.J|=l 
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Proof. We have 



di- di = ^ di- ^ di < ^ di < ^ di 

i-.Si&T i-.tiGT i:si£T,ti^T i:UeT,Si^T i:SieT,ti^T i:\Tn{u^,v^}\=l 



Now using the assumption of the lemma we have 



f{T)-a Yl ^ f{T)-aY,^iv) < 0. 

i:\Tr\{ui,Vi}\=l 



(2) 



Since the function / is non-negative, it must be that X]r|Tn{ui «i}|=i ^« ^ ^- Rearranging the terms, 
we get f{T)/J2r.\Tn{u^M\=i < a. □ 
Assuming that the input instance is feasible, let U* be a set with size m = \U*\, separated 
demand D* = T.i:\U'n{u,,v,}\=idi, and value f{U*)/D* < B. 

Lemma 3.6 In one iteration of the outer loop of Algorithm [II the probability that 
1 ^ is at least " 



Proof. Let e = Y^^. We denote by A the event that \U* D S\ > f (1 + where S is the 
random set chosen by Algorithm [H and bound the above probability by the following product: 



Pr 



> Pr 



A 



■Pr[A]. 



We observe that by Theorem 12.21 t^ie probability of A is at least ^n"^/^. All the probabilities and 
expectations in the rest of the proof are conditioned on the event A. 

Let us now consider the expected value of "^^^u* w{v). Fix a particular demand pair {ui,Vi} 
that is separated by the optimal solution, and assume without loss of generality that Uj G U* and 
Vi ^ U* . Let pu be the probability that Ui G S, and py be the probability that Vi G S. Then 
p„ = > (1 + £)/2, Pi. = |, and the two events are independent. So 

Pr[ui = Si] = Fi[ui eSAv.^S] = Pu-{1- Pv) > (1 + e)/4, 
Pr[w, = ii] = Pr[ni ^ 5 A G 5] = (1 - p„) • p„ < (1 - e)/4. 

Then the expected contribution of this demand pair to Xl^eC/* '^{'^) is equal to 

Pr[nj = Si] ■ di + Pr[ui = tj] • {-di) > di - ^. 



By linearity of expectation, 

E E^(^) 

.1)6(7* 

We now use Markov's inequality j30j to bound the desired probability. For this we define a non- 
negative random variable Y = D* — X^t^eC/* '"^(^)- Then E[y] < (1 — e/2)D* . So 



> D* ■-. 
2 



Pr 



Pr 



y > (1 - -)L>* 

~ 4 



< 



E[y] 



{l-e/A)D* - l-e/4 



4-e - 4 
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It follows that 



Pr 



e 1 /Inn 1 
> - = -W > 



4 4V n 



concluding the proof of the lemma. □ 

Theorem 3.7 For any feasible instance of SSC problem, AlgorithmUl returns a solution of cost at 
most 4y^[^ • B, with probability at least p. 

Proof. By Lemma 13.61 the inequality XlueC/* ^i''^) — ^* ' \ \f^' holds with probability at least 

c/8n^ in each iteration. Then the probability that it holds in any of the In(Y^) iterations is at 
least p. Now, assuming that it does hold, the algorithm finds a set T such that 

fiT)-a-Y^w{v) < f{Un-a-J2Mv) < /(f^*) " (^a/I^ " ^) f ^* < 



Applying Lemma ESI we get that Vi}\=i di < a = ■ B, which means that T 

is the required approximate solution. □ 

3.3 Submodular balanced cut 

For submodular balanced cut, we use as a subroutine the weighted SSC problem that can be 
approximated to a factor 7 = {s/t^) using Algorithm [TJ This allows us to obtain a bicriteria 
approximation for SBC in a similar way that Leighton and Rao [28] use their algorithm for sparsest 
cut on graphs to approximate balanced cut on graphs. Leighton and Rao present two versions of 
an algorithm for the balanced cut problem on graphs — one for undirected graphs, and one for 
directed graphs. The algorithm for undirected graphs has a better balance guarantee. We describe 
adaptations of these algorithms to the submodular version of the balanced cut problem. Our first 
algorithm extends the one for undirected graphs, and it works for symmetric submodular functions. 
For a given b' < 1/3, it finds a ^'-balanced cut whose cost is within a factor O ^f^) °^ ^^"^ 
of any ^-balanced cut, for b' < 6 < ^. The second algorithm works for arbitrary non-negative 
submodular functions and produces a 6'/2-balanced cut of cost within O ^jry) °^ 6-balanced 
cut, for any b' and b with b' < b < 1/2. 

3.3.1 Algorithm for symmetric functions 

The algorithm for SBC on symmetric functions (Algorithm [2]) repeatedly finds approximate weighted 
submodular sparsest cuts {Si, Si) and collects their smaller sides into the set T, until (T, T) becomes 
^'-balanced. The algorithm and analysis basically follow Leighton and Rao [28], with the main dif- 
ference being that instead of removing parts of the graph, we set the weights of the corresponding 
elements to zero. Then the obtained sets Si are not necessarily disjoint. 



Theorem 3.8 // the system {V,f,w), where f is a symmetric submodular function, contains a 
b-balanced cut of cost B, then Algorithmic finds ol b' -balanced cut T with f{T) = O 
for a given b' < b, b' < I. 
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Algorithm 2 Submodular balanced cut for symmetric fmictions. Input: V, f, w, b' < ^ 

1: Initialize w' = w, i = 0, T = ^ 
2: while w'{V) > (1 - b')w{V) do 

3: Let 5 be a 7-approximate weighted SSC on V, f, and weights w' 
4: Let 5i = argmin('u;'(5),w;'(S)); w'{Si)^0; T^TuSi; + 1 

5: end while 
6: return T 



Proof. The algorithm terminates in 0{n) iterations, since the weight of at least one new element 
is set to zero on lined] (otherwise the solution to SSC found on line [3] would have infinite cost). 

Now we consider w{T). By the termination condition of the while loop, we know that when it 
exits, w'(y) < (1 — b')w{V), which means that w' has been set to zero for elements of total weight 
at least b'w{V). But those are exactly the elements in T, so w{T) > b'w{V). Now consider the 
last iteration of the loop. At the beginning of this iteration, we have w'{V) > (1 — b')w{V), which 
means that at the end of it we have w'{V) > ^(1 — b')w{V), because the weight of the smaller 
(according to w') of S or S is set to zero. But w'{V) at the end of the algorithm is exactly the 
weight of T, which means that w{T) > ^(1 — b')w{V) > ^w{V) > b'w{V), using the assumption 
b' < 1/3 twice. So the cut (T,T) is ^'-balanced. 

Suppose that U* is a 6-balanced cut with f{U*) = B. In any iteration i of the while loop, 
we know that two inequalities hold: w'(U*) + w'{U*) > (1 — b')w{V) (by the loop condition), and 
max{w' {U*),w' (U*)) < (1 — b)w{V) (by 6-balance). Given these inequalities, the minimum value 
that the product w'{U*) ■ w'{U*) can have is (6 — b')w{V) ■ (1 — b)w{V). So with weights w' , there 
is a solution to the SSC problem with value 

f{U*) < B 

w'{U*)w'{U*) - {b - b')w{V) ■ {I - b)w{Vy 

and the set Si found by the 7-approximation algorithm satisfies 

f{Si) ^ jB 

w'{Si)w'{Si) - {b - b')w{V) ■ {I - b)w{Vy 

Since in iteration i, w'{Si) = w{Si \ Uj=o w'{Si) < w{V), and (1 — 6) > 1/2, 

Now /(T) < f{Si) < w{T) ■ 2Bj/{b - b')wiV) = B ■ Oi^). □ 
3.3.2 Algorithm for general functions 

The algorithm for general functions (Algorithm [3|) also repeatedly finds weighted submodular spars- 
est cuts {Si, Si), but it uses them to collect two sets: either it puts Si into Ti, or it puts Si into 
T2. Thus, the values of /(Ti) and /(T2) can be bounded using the guarantee of the SSC algorithm 
(where f{S) = f{S)). 
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Algorithm 3 Submodular balanced cut. Input: V, f, w, b' 
1: Initialize w' = w , i = 0, Ti = T2 = 9 
2: while w'{V) > (1 - b')w{V) do 

3: Let Si be a 7-approximate weighted SSC on V, /, and weights w' 
4: if w'{S^) < w'{Si) then set_ri ^ Ti U Sf, w'{Si) ^0; + 1 

5: else set T2 ^ r2 U Si; w'{Si)^0; + 1 

6; end while 

7; if w{Ti) > w{T2) then return Ti else return T2 



Theorem 3.9 // the system (y,f,w) contains a b-balanced cut of cost B, then Algorithmic finds 
a b' / 2 -balanced cut T with f{T) = O for a given b' < b. 

Proof. When the while loop exits, w'{V) < {l — b')w{V), so the total weight of elements in Ti and 
T2 (the ones for which w' has been set to zero) is at least b'w{V). So max(?i;(ri), t(;(r2)) > b'w{V)/2. 
At the beginning of the last iteration of the loop, w'{V) > (1 — b')w{V). Since the weight of the 
smaller of Si and Si is set to zero, at the end of this iteration w'{V) > ^(1 — b')w{V). Let T be 
the set output by the algorithm. Since w'{T) = 0, we have w{T) > w'(y) > ^(1 — b')w{V) > b' /2, 
using b' < 1/2. Thus we have shown that Algorithm [3] outputs a 6'/2-balanced cut. 

The function values can be bounded as f{Ti) = B ■ 0(^^) and f{T2) = B ■ 0{j^) using a 
proof similar to that of Theorem 13. 8[ □ 

4 Submodular minimization with cardinality lower bound 

We start with the lower bound result. Let i? be a random subset of V of size a = let /3 = 
and X be any parameter satisfying = uj(lnn) and such that a and /3 are integer. We use the 
following two monotone submodular functions: 

fsiS) = min(|5|,a), f^{S) = min (/3 + |5 n \S\, a). (3) 

Lemma 4.1 Any algorithm that makes a polynomial number of oracle queries has probability n~'^^^^ 
of distinguishing the functions /s and f^ above. 

Proof. By Lemma l2.H it suffices to prove that for any set S*, the probability that fz{S) 7^ fiiS) 
is at most n~^^^\ It is easy to check (similarly to the proof of Lemma l3.2p that Pr[/3(S') 7^ /4('S')] 
is maximized for sets S of size a. And for a set S with |5| = a, fsi^S) ^ fiiS) if and only if 
(3 + \S (1 R\ < \S\, or, equivalently, 15 H i?| > f3. So we analyze the probability that 15" n i2| > /3. 

i? is a random subset of V of size a. Let us consider a different set, R' , which is obtained by 
independently including each element of V with probability a/n. The expected size of R' is a, and 
the probability that = a is at least l/(n + 1). Then 

Pv[\Sr\R\>/3] = Fv[\Sr\R'\> /3 \ \R'\ = a] < (n + 1) • Pr [|5 n > /?] , 

and it suffices to show that Pr [|S'ni?'| > /?] = n~'^^^\ For this, we use Chernoff bounds. The 
expectation of |5 n R'\ is /i = a\S\/n = o? jn = x^/25. Then /3 = 5/i. Let (5 = 4. Then 

PrOSni?|>(l + «)^] < ((j-;^ 
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Since = a; (Inn), we get that this probabihty is n '^^^^ □ 

Theorem 4.2 There is no (p, a) bicriteria approximation algorithm for the SML problem, even 
with monotone functions, for any p and a with ^ = o ( y^j^) • 

Proof. We assume that any algorithm for this problem outputs a set of elements as well as the 
function value on this set. Suppose that a bicriteria algorithm with ^ = o (y^l^) exists. Let f^ 

and /4 be the two monotone functions in with x = -3^, where 5 > 1 is a constant that ensures 
that a and (3 are integer. Then x satisfies = a;(lnn). Consider the output of the algorithm when 
given /4 as input and W = a. The optimal solution in this case is the set R, with f{R) = (3. So 
the algorithm finds an approximate solution T with fi{T) < pj3 and |r| > aa. However, we show 
that no set S with f^{S) < pf3 and \S\ > aa exists, which means that if the input is the function 
/a, then the algorithm produces a different answer, thus distinguishing f^ and /4. We assume 
for contradiction that such a set 5 exists and consider two cases. First, suppose |5| > a. Then 
fsiS) < pP = ~ ^ since 6 > 1 and by definition cr < 1. But this is a contradiction 

because fz{S) = a for all S with \S\ > a. The second case is jS"! < a. Then we have |5| > aa and 
f?,[S) < pfi = ^ < \S\/ 8, which is also a contradiction because |5| > aa > and fs^S) = \S\ for 
\S\<a. □ 



4.1 Algorithm for SML 

Our relaxed decision procedure for the SML problem with weights {0, 1} (Algorithm H]) builds up 
the solution out of multiple sets that it finds using submodular function minimization. If the weight 
requirement W is larger than half the total weight w{V), then collecting sets whose ratio of function 
value to weight of new elements is low (less than 2B/W), until a total weight of at least W/2 is 
collected, finds the required approximate solution. In the other case, if W is less than w{V)/2, the 
algorithm looks for sets Tj with low ratio of function value to the weight of new elements in the 
intersection of Ti and a random set Si. These sets not only have small f{Ti)/w{Ti) ratio, but also 
have bounded function value f{Ti). If such a set is found, then it is added to the solution. 

Theorem 4.3 Algorithm ^is a (Sy^jj^, ^) bicriteria decision procedure for the SML problem. 
That is, given a feasible instance, it outputs a set U with f{U) < 5\/~^^B and wiU) > W/2 with 
probability at least p. 

Proof. Assume that the instance is feasible, and let C/* C 1/ be a set with w{U*) > W and 
f{U*) < B. We consider two cases, W > w{V)/2 and W < w{V)/2, which the algorithm handles 
separately. 

First, assume that W > w{V)/2 and consider one of the iterations of the while loop on line[3j 
By the loop condition, w{Ui) < W/2, so w{U* \ Ui) > W/2. As a result, for the set U* , the 
expression on line H] is negative: 

f{U*)-'^-w{U*\U,) < f{U*)-B < 0. 

Then for the set Tj which minimizes this expression, it would also be negative, implying that 
uj{Ti \ Ui) is positive, and so w{Ui) increases in each iteration. As a result, if the instance is 
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Algorithm 4 SML. Input: V, f,w:V ^ {0, 1}, W, B, p 



Initialize Uq = v>; i = 

i{W> w{V)/2 then > case W > ^ 

while w{Ui) < W/2 do 

Let Ti be a subset of V minimizing /(T) — ^ • w{T \ Ui) 
if f{Ti) < ^ • w{Ti \ Ui) then Let Ui+i = UiUTi; i = i + l else return fail 
end while 
return U = Ui 
end if 

Leta = f > casel^<^ 

while w{Ui) < W/2 do 

Choose a random 5^ C F \ f/j, including each element with probability J^-^ 

Let Ti be a subset of V minimizing f{T) — a ■ w{T Si) 

if f{Ti) < a ■ w{Ti n 5,) and f{Ti) < 45 then Let Ui+i = UiUTi; i = i + l 

if the number of iterations exceeds -^"^^J^ In (^j^^ > return /aiZ 
end while 
return U = Ui 



feasible, then after at most n iterations of the loop on line[3l a set U is found with w{U) > W/2. 
For the function value, we have 

f{u) < Y^/m) < ^I^MTAc^O < ^-HV) < AB 

i i 

by our assumption about W. 

The second case is < w{V)/2. Assuming Claim [53] below, which is proved later, we show 
that in each iteration of the while loop on line \T0\ with probability at least ^^7/2 ■> & i^gw non- 
empty set Ti is added to U. This implies that after ^"'^J In ^ iterations, the loop successfully 
terminates with probability at least p. 

Claim 4.4 In each iteration of the while loop on line{T^ of Algorithm^ both of the following two 
inequalities hold with probability at least ^^/2 ■ 



B W Inn 

wiU*nSi) > - = —J— and w{U*nSi) < 1.5W. (4) 
a 2 \ n 

We show that if inequalities (jH hold, then the set Tj found by the algorithm on line [12] is 
non-empty and satisfies the conditions on line 1131 which means that new elements are added to U. 
Since Tj is a minimizer of the expression on line 1121 using @, 

f{T^)-a■w(T^nS^) < f{U*)-a■w{U*nS^) < f{U*)-B < 0, 

which means that Tj satisfies the first condition on line 1131 and is non-empty. Moreover, from the 
same inequality and the second part of (jl]) we have 



f(Ti) < f(U*) + a-(w(TinSi)-w(U*nSi)) < B + a-w(U*nSi) < B + 1.5aW < AbJ-^, 

V mn 
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which means that Ti also satisfies the second condition on line 1131 

Now we analyze the function value of the set output by the algorithm. Let Tj be the last set 
added to U by the while loop, and consider the set Ui just before Tj is added to it to produce C/j+i. 
By the loop condition, we have w{Ui) < W/2. Then, by submodularity and condition on line [T3| 



W 



j=0 



j=0 



So for the set U that the algorithm outputs, f{U) < f{Ui) + f{Ti) < 5By^. And by the exiting 
condition of the while loop, w{U) > W/2. □ 



Proof of Claim 14.41 Because the events corresponding to the two inequalities are independent, 
we bound their probabilities separately and then multiply. To bound the probability of the first 
one let m = w{U* \ Ui) be the number of elements of U* with weight 1 that are in 1/ \ Ui. Since 
w{U*) > W and w{Ui) < W/2 by the condition of the loop, we have m > W/2. We invoke Theorem 



with parameters m, q = W/w{V), and e 



w{V) 
2m 



To ensure that e < 



l-q 



and this theorem 



can be applied, we assume that n > 9, so that y^ln n/n < 1/2, and get 



w{V) 
W 



w{V) 
2m 



Inn w{V) 
~ ^ 2W 



1 > 0. 



Thus the inequality w{U* D Si) > \qm{l + e)] > qme = ^ holds with probability at least 
(simplifying using inequalities w{V) — W> w{V)/2, w{V) < n, and 1 <W < 2m) 



cqm 2 exp 



-e^qm 



W 

w{V) 



m 



exp 



w{y) W m Inn 



^m^ n w{V) {w{V) - W) 



> cn 



-7/2 



W 



For the second inequality, we notice that the expectation of w{U* n Si) is w{U* j • ;^yy 
So by Markov's inequality, the probability that w{U* n Si) < 1.5W is at least 1/3. 



< W. 

□ 



5 Submodular load balancing 
5.1 Lower bound 

We give two monotone submodular functions that are hard to distinguish, but whose value of the 
optimal solution to the SLB problem differs by a large factor. These functions are: 



fbiS) = minds'!, a) feiS) = min 




Here {V^} is a random (unknown to the algorithm) partition of V into m equal-sized sets. We set 

m = ^ ~ m~ ^^5^' 1^ ~ T'' "^i^^ ^'^y parameter x satisfying = a;(lnn), and values chosen 
so that a and /3 are integer. 

Lemma 5.1 Any algorithm that makes a polynomial number of oracle queries has probability n~'^^^') 
of distinguishing the functions /s and /g above. 
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Proof. By Lemma 12.11 it suffices to bound the probability, over the random choice of the sets 
Vi, that f^iS) 7^ feiS) for any one set S. Since fz'> f^-, this is the same as Pr {f&{S) — f^{S) < 0]. 
First, we show that this probability is maximized when \S\ = a. For IS"! > a, 



Pr[/6(5)-/5(S) <0] = Pr 



^min(/3,|Snyi|) < 



a 



and since the sum in this expression can only decrease if an element is removed from 5, we have 
that for \S\ > a, this probability is maximized at \S\ = a. For IS"! < a. 



Pr[/6(5)-/5(5) <0] = Pr 



mm I min {f3,\S r^Vi\) ,a 



|5| <0 



Pr 



Pr 



min ^^min(/3, |5n -^l^nVil, a-\S\^ 



< 



min(/3 - \SnVi\,0) < 



Since the sum in this expression can only decrease if an element is added to S, we have that for 
l^l < a, the probability is maximized at \S\ = a. 

So suppose that \S\ = a. We notice that if for all i, \SriVi\ < /3, then /5(-S') = feiS). Therefore, 
a necessary condition for the two functions to be different is that ISn > /3 for some i. Since Vi 
is a random subset of V of size a, we can use the same calculation as in the proof of Lemma l4.1l to 
show that Pr [|S'n T^lI > /3] < n~'^^^\ Applying the union bound, we get that the probability that 
n > /3 for any i is also n~'^^^\ □ 

Theorem 5.2 The SLB problem is hard to approximate to a factor of a (a/j^) • 



Proof. Suppose that there is a 7-approximation algorithm for SLB, where 7 = 
X = yjnj^^^ where (5 > 1 is such that a and /3 are integer. This satisfies x? = a;(lnn). Now consider 
running the algorithm with the input function /g and size of partition m. For this input, partition 
{Vi} constitutes the optimal solution whose value is f%(yi) = /3, so the algorithm returns a solution 
whose value is at most 7/3 = a/5. However, for the input /s and m, any partition must contain a 
set S with size \S\ > n/m = a (since this is the average size). For this set, the function value is 
fb{S) = OL > a/ 6. This means that for /s the algorithm produces a different answer than for /g, 
which contradicts Lemma |5. 11 □ 



5.2 Algorithms for SLB 

We note that the technique of Svitkina and Tardos [37] used for min-max multiway cut can be 
applied to the non-uniform SLB problem to obtain an 0{y/n logn) approximation algorithm, using 
the approximation algorithm for the SML problem presented in Section H] as a subroutine. Also, 
an 0{^/nlogn) approximation for the non-uniform SLB appears in |13j . 

In this section we present two algorithms, with improved approximation ratios, for the uniform 
SLB problem. We begin by presenting a very simple algorithm that gives a min(m, \^~\ ) = 0{^/n) 
approximation. Then we give a more complex algorithm that improves the approximation ratio to 
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O ( Y^i^) ) thus matching the lower bound. Our first algorithm simply partitions the elements into 
m sets of roughly equal size. 

Theorem 5.3 The algorithm that partitions the elements into m arbitrary sets of size at most j"^] 
each is a min(m, |"^] ) approximation for the SLB problem. 

Proof. Let {Uf, U^} denote the optimal solution with value B, and let A be the value of the 
solution {Si, Sm} found by the algorithm. We exhibit two lower bounds on B and two upper 
bounds on A, and then establish the approximation ratio by comparing these bounds. For the 
lower bounds on B, we claim that B > maxj^v /({j}) and ^ ^ f(^)/i^- For the first one, let j 
be the element maximizing f{{j}), and let U* be the set in the optimal solution that contains j. 
Then B > f{U*) > /({j}) by monotonicity. For the second bound, by submodularity we have that 
f{V) < f{U*) < niB. To bound A, we notice that A < f{V) (by monotonicity), and that A < 
1"^] maxjgy /({j}), since each set Si contains at most j"^] elements, and f{Si) < X^jg^. /({j})- 
Comparing with the lower bounds on B, we get the result. □ 



Algorithm 5 Submodular load balancing. Input: V, m > yi^' monotone /, B, p 
1: if for any v £ V, f{{v}) > B, return fail 
2: Let a = Bm/\/n Inn; Initialize V' = V, i = 
3: while \ V'\ > rn-s/^ do 

4; Choose a random 5 C y, including each element independently with probability ^1^, ^ 
5: if \S\ < 2^ then 

6: Let T C 5 be a subset minimizing f{T) — a ■ \T\ 

7: if f{T) - a • |r| < then set = T; i = i + 1; V' = V'\T 

8; end if 

9: if the number of iterations exceeds In(Y^), return fail 
10: end while 

11: Let T be the collection of sets Tj produced by the while loop 

12: Partition T into m groups 71, ...,Tm, such that Yli-T eT 1-^*1 — "^m each Tj 

13: Let Ui, Um be any partition of V with each set of size at most y^I^ 

14: For each j £ {1, m}, let Vj = Uj U Ur,e7; 

15: return {Vi, ...,Vm} 



For the more complex Algorithm [U we assume that m > 2 y/j^, because for lower values of m 
the above simple algorithm gives the desired approximation. Also, the simple algorithm has better 
guarantee for all n < e^^ , so when analyzing Algorithm [5l we can assume that n is sufficiently large 
for certain inequalities to hold, such as In^ n < n. The algorithm finds small disjoint sets of elements 
that have low ratio of function value to size. Once a sufficient number of elements is grouped into 
such low-ratio sets, these sets are combined to form m final sets of the partition, while adding a 
few remaining elements. These final sets have roughly n/m elements each, so using submodularity 
and the low ratio property, we can bound the function value for each set in the partition. 

First we describe how some of the steps of algorithm work. The loop condition > my^j^ 
and our assumptions m > 2 ^/j^ and In^ n < n imply that the probability (used on line U]) is 

less than one. The partition on line 1131 can be found because at this point, the size of V is at most 
m-v/r^. For the partitioning done on line 1121 we note that since each Tj is a subset of a sample set 
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S with IS*! < 2n/m, it holds that jT'tl < 2n/m. Also, the total number of elements contained in all 
sets Tj is at most n (since they are disjoint). So a simple greedy procedure that adds the sets Tj to 
Tj in arbitrary order, until the total number of elements is at least n/m, will produce at most m 
groups, each with at most 3n/m elements. 

Theorem 5.4 // given a feasible instance of the SLB problem, Algorithmic outputs a solution of 
value at most ^y^i^ • B with probability at least p. 

Proof. By monotonicity of /, the algorithm exits on line [1] only if the instance is infeasible. 
Assume that the instance is feasible and let {U^, . . . , U^} denote a solution with maxj f{U*) < B. 
We consider one iteration of the while loop and show that with probability at least it finds a 
set T C 5 satisfying /(T) — a ■ \T\ < 0. Then the probability that the size of V' is reduced to 
rriy^ after ^ln(Y^) iterations is at least p. 

Assume, without loss of generality, that U* \s the set that maximizes \U* n V'\ for this iteration 
of the loop. If we let n' = \V'\, then \Ul n V'\ > \n' /ni\. Suppose the sample S found by the 
algorithm has size at most 2n/m, and let i = n 5*1 denote the size of the overlap of S and U^. 
By monotonicity of /, we know that f [111(^3) < f{Ul) < B. Since the algorithm finds a set T C 
minimizing the expression f{T) — a\T\, we know that the value of this expression for T is at most 
that for UlnS: 

fiT)-a\T\ < f{UlnS)-a\UlnS\ < B - ^^=. 

In order to have f{T) — a\T\ < 0, we need t > . Next we show that the event that both 

t > ^"J^"" and l^l < 2n/m happens with probability at least 

Let X = To bound the probability that t > X, we focus on an arbitrary fixed subset of 

UlCiV of size [n'/m] (which is possible because |C/j*ny| > [n'/m]), and compute the probability 
that exactly [x] elements from this subset make it into the sample S. In particular, this is the 
probability that sampling [n'/m] items independently, with probability n/mn' each, produces a 
sample of size [j;]. We note that x S (l,n'/m), so [x] is a valid sample size. These bounds follow 
because inside the while loop, m < n' y^ln n/n < Vnlnn, so x > 1. Also, n'/m > y^n/lnn > Inn > 
y/n Inn/m by the loop condition and our assumptions on n and m, so 2: < n'/m. Let 7, 5 G [1, 2) 
be such that ^ • n! /m = \n' /m~\ and 6 ■ x = [x]. We use an approximation derived from Stirling's 
formula as in the proof of Theorem 12.21 



Pr[t = [x]] > 



> 



^n' /m\ ( n n — 

6x 



c 



c 
n 



ml \ mn' ) \ mn' ) 



jn' 




-5x 



(6) 
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> ^•(— a/— 



where the last inequahty comes from observing that om assmnption of m > 2^j^, together with 
7/5 < 2, imply that the last term on line ^ is greater than 1. 

If we take a derivative of this bound with respect to m, which is 



dm 



7 



n V ^'<T^ V In n 



(5 V?i In 71 ^ 



n 



/ 7 / n 



m^ 



5\^n \a n 



\5m V Inn 



\dm V Inn ' ~'~ 



and set it to zero, we find that the bound is minimized when m = ^\f^^- Substituting this value, 



Pr[t = [rr]] > c-n ^ > c-n 2 >c-n^. 

To bound the second probability, that IS*] < 2n/m, we note that E [IS"!] = n/m and use Chernoff 
bound as well as the loop condition that implies m < n\/^^ < \/nlnn. 



Pr 



\S\>2^ 
m 



< ',4 



^ I C \ V Inn 

.4/ 



If n is sufficiently large that (|)^ < we can use the union bound to get 



Pr 



t>x and IS] < 2 



n 



m 



> 



n' 



2n2 



c 

2^' 



This establishes that on feasible instances, the algorithm successfully terminates with probability 
at least p. Let us now consider the function value on any of the sets Vj output by the algorithm. 
By submodularity, 



For each Ti we know that /(Tj) < a - |Tj|, and by the check performed on linedl we have f{{v}) < B 
for each v £ V. Using this and the bounds on set sizes. 



f{V,) < B 



n 
Inn 



+ aY,\Ti\ < B 



n 3n ^ 

+a =B 

Inn m 



n m 3n\ ^ n ^ 

Inn Vnlnn rn J \ Inn 



□ 



6 Approximating submodular functions everywhere 

We present a lower bound for the problem of approximating submodular functions everywhere, 
which holds even for the special case of monotone functions. We use the same functions ([3]) as for 
the SML lower bound in Section HI 

Theorem 6.1 Any algorithm that makes a polynomial number of oracle queries cannot approxi- 
mate monotone submodular functions to a factor o ( y^j^) ■ 
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Proof. Suppose that there is a 7-approximation algorithm for the problem, with 7 = ( y^j^) ; 
which makes a polynomial nmnber of oracle queries. Let x = ^Jnjb^^ which satisfies = u;(lnn). 
By Lemma |4.1|, with high probability this algorithm produces the same output (say /) if given as 
input either or /4. Thus, by the algorithm's guarantee, / is simultaneously a 7-approximation 
for both /s and f^. For the set R used in /4, this guarantee implies that fsiR) < 7/(-R) < ^fi{R). 
Since fsiR) = a and /^{R) = /3, we have that 7 > a//3 = \/nlx = 27, which is a contradiction. □ 



6.1 Approximating monotone two-partition submodular functions 

Recall that a 2P function is one for which there is a set i? C 1/ such that the value of /(S*) depends 
only on |S'n i?| and IS'H Our algorithm for approximating monotone 2P functions everywhere 
(Algorithm |6]) uses the following observation. 

Lemma 6.2 Given two sets S and T such that \S\ = \T\, but f{S) 7^ f{T), a 2P function can be 
found exactly using a polynomial number of oracle queries. 

Proof. This is done by inferring what the set R is. Using S and T, we find two sets which differ 
by exactly one element and have different function values. Fix an ordering of the elements of S, 
{si, Sfc}, and an ordering of elements of T, {ti, ...,tk}, such that the elements of S CiT appear 
last in both orderings, and in the same sequence. Let Sq = S, and Si be the set S with the first i 
elements replaced by the first i elements of T: Si = {ti, tj, Sj+i, Sfc}. Evaluate / on each of 
the sets Si in order, until the first time that f{Si-i) ^ fiSi). Such an i must exist since Sk = T, 
and by assumption /(T) ^ f{S). Let U = {ti, Sj+i, s^}, so that Si-i = UU {si} and 

Si = uu{ti}. 

The fact that f{U U {sj}) / f{U U {ti}) tells us that either Si G R and ti ^ R, or vice versa. 
Without loss of generality, we assume the former (since the names of R and R can be interchanged). 
Now all elements 'mV\U can be classified as belonging or not belonging to R. In particular, if for 
some element j G U, f{U U {j}) = f{U U {si}), then j G R; otherwise f{U U {j}) = f{U U {ti}), 
and j ^ R. To test an element u £ U, evaluate f{U — {u} + {si,ti}). This is the set Si-i with 
element u replaced by tj. If u G R, then replacing one element from R by another will have no 
effect on the function value, and it will be equal to f{Si-i). If n G i?, the we have replaced an 
element from R by an element from R, and we know that this changes the function value to f{Si). 
So all elements of V can be tested for their membership in R, and then all function values can be 
obtained by querying sets W with all possible values of |VK n i?| and |VF fl ^|. □ 

Algorithm 6 Approximating a monotone 2P function everywhere. Input: V,f,p 
1: Query values of /(0), fiV), and f{{j}) for each j G V 

2; For each i G {2, ...,n — 1}, independently generate n^'^ln ^ir^^ random sets by including each 

element of V into each set with probability ^. Query the function value for each of these sets. 
3; If the previous two steps produce any two sets Si and ^2 with = |52| and f{Si) / f{S2), 
then find the function exactly, as described in Lemma |6.2[ 

r/(0) if 5 = 



Else, let j G V he an arbitrary element, and output f{S) 



fi{j}) if 1 < |5| < 2^ 
^ if |5| > 2V^ 
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Theorem 6.3 With probability at least p, the function f returned by Algorithm satisfies 
f{S) < f{S) < 2^ ■ f{S) for all sets S<ZV. 

Proof. If the algorithm finds two sets Si and 52 such that |5i| = |52| and f{Si) ^ /(5'2) 
during the sampHng stage (steps 1 and 2), then the correctness of the output is imphed by Lemma 
16. 2i If it does not find such sets, then it outputs the function / shown in step 4. It obviously 
satisfies the inequality for the case that 5 = 0. For the case that 1 < |5| < we observe that 

if the algorithm reaches step 4, it must be that the value of / is identical for all singleton sets, 
i.e. f{{j}) = f{{j'}) for all G V. Now, f{S) > f{{j}) = f{S) by monotonicity. Also, by 
submodularity, f{S) < Yljes /({i}) ~ 1*^1 " fi'^) — • /(5), establishing the correctness for the 
case that |5| < 2y/n. For the last case, |5| > 2^, the inequality f{S) < f{V) = 2^-f{S) follows 
by monotonicity. For the other one, f{S) < f{S), we need an additional nontrivial lemma. 

Since the 2P function f{S) depends only on two values, \S n R\ and |5 n let us denote by 
f{k, I) the value of the function / on a set S with |5ni?| = k and |5n-R| = /. We say that such a set 
S corresponds to the pair {k,l). We assume that < < n, because if \R\ = or \R\ = n, then 
f{S) is a function that depends only on |5|, and it equally well can be represented as a 2P function 
with any other set R. Furthermore, we assume without loss of generality that |-R| < (otherwise 
interchange R and R), and let K = \R\ and L = \R\ (which are not known to the algorithm). 

Lemma 6.4 For any k and any I, f{k,0) > 4if(^) /(O, > 4ifi^)- 

Using this lemma to finish the proof, let k = \S Ci R\ and / = |5 n We observe that by 
monotonicity, f{S) > f{k,0) and f{S) > /(O,/). Moreover, since |5| = A; + / > 2^/n, we have 
max(A:, /) > ^. So by Lemma [M f{S) > 2^^/(1^) > §^ = f{S). □ 

The proof of Lemma 16.41 is involved, and we first sketch the main ideas. We call a pair {k,l) 
balanced if k/l is close to K/L. Then, with significant probability, the algorithm samples sets 
corresponding to all balanced pairs. Since the algorithm checks for sets of the same size with 
different function values, we can assume that if it proceeds to step 4, then for sets S corresponding 
to balanced pairs, f{S) is a function F that depends only on |5|. We use submodularity to show 
that F is concave. Then we decompose f{k,0) as X^i=i[/(^,0) — f{i — 1,0)] and lower-bound each 
term in this sum separately by comparing it to an increment f{i,j) — f{i — l,j) for some j with 
(i,j) balanced. Then, using concavity of F, we lower-bound their sum. 

To prove Lemma l6.4( we use a definition and several preliminary lemmas. 

Definition 6.5 A pair of integers {k, I) with k < K and I < L is said to be balanced if it satisfies 

K K 

I 2 < k < 1- — + 2. (7) 

L L 

Intuitively, in a set corresponding to a balanced pair, the numbers of elements from R and R 
are proportional to the sizes of the two sets (see Figured]). 

Lemma 6.6 Suppose that m < n elements are selected independently with probability g € ^^^] 
each, and let X denote the total number of selected elements. Then for any integer x G [0,m — 1], 

J_ ^ Ft[X = x + 1] ^ 
Pi[X = x] 
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Proof. 



Pv[X = x + l] 



\x+l 



Pt\X 



x\ 



) 9"+^ (1 - g) 



m—x—l 



with the minimum value of l/m(n — 1) > 1/n? achieved at x 
value of m(n — 1) < achieved at x = and q = 



{m — x) q 
" {x + l){l-q) 

m—1 and q = 



and the maximum 

□ 



Lemma 6.7 If Algorithmic reaches step 4, then with probability at least p, for all balanced {ki,li) 
and {k2-,h) such that ki + li = k2 + h, it holds that f{ki,li) = f{k2,l2)- In other words, for all 
balanced pairs {k,l), the value of f{k,l) depends only onk + l. 

Proof. The lemma follows if we show that with probability at least p, for each balanced (/c, /) 
with k + I < n, the algorithm samples at least one set S corresponding to {k,l). This is because 
the algorithm verifies that the function value for the sets that it samples depends only on the set 
size. 

So consider a specific balanced pair (k, I) and one random set S 
generated by the iteration i = k + l of step 2 of the algorithm. The ^ 
probability of sampling each element in this iteration is q = ^ = 
Using ^ and its equivalent {k -2)L/K <l < {k + 2)L/K, 
we see that this probability satisfies the following: 



< 



< 



2L 



k 

Kn 



and — 

L n 



< 



I 2 
< 7 + -. 

L n 



k 2L 

So the expected value of jS* n i?| is qK £ [k — 2L/n, k + 2L/n] C 
[k—2, k+2]. Similarly, the expected value of |5n^| is qL G [1 — 2, 1 + 
2]. Let fik be the most likely number of sampled elements when 
independently sampling K elements with probability q each. Then 
fik is equal to either [qK\ or \qK~\ . From above considerations and 
because k is an integer, we have that /u^ G [fc — 2,/c + 2]. Now, 
since Hk is the most likely value, we know that Pr[|5 Ci R\ = fik] > 
1/{K + 1) > 1/n. By Lemma ES] (with m = K), 

-2-|fc-Mfcl ^ 
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Figure 1: In the table of pairs 
(fc,/), the shaded cells corre- 
spond to balanced pairs, and 
the -fC-biased (dashed) and 
L-biased (dotted) walks are 
shown. 



We similarly define /x/, observe that G — 2, / + 2], and conclude 
that Pr[|5 n -R| = ^] > . Since the two events are indepen- 
dent, the probability that both of them occur, and thus that S 
corresponds to (fc, /), is at least n~^^ . 

We observe that for any z, there are at most four balanced pairs (fc, I) such that k + l = i. This 
is because if some pair (A;, I) satisfies ([7]), then the pair (fc — 4, ^ + 4) doesn't satisfy it: 



k-A < 



K 

I— + 2 

1j 



K K 
l—-2< (/ + 4)— -2. 



So there is a total of at most An pairs {k, I) for which we would like the algorithm to sample their 
corresponding sets. Since the number of trials for each value of A; + / is n^*^ In {^^z:^ i the probability 

/) is not sampled is at most 
1 — p 



that a set corresponding to any particular pair 

In 



1 



n 



-lO^ 



An " 
1"P/ 



< 



-ln( 



4n 
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Since there are at most 4n pairs of interest, by union bound we have that the probabihty that at 
least one of them remains unsampled is at most {1 — p). □ 

Suppose the condition in Lemma 16.71 holds. Let us define a function F{i) to be equal to f{k,l) 
such that k + 1 = i and (/c, I) is balanced. F{i) is defined for all i £ {0, ■■■,n}, since for any such i 
there is at least one balanced pair (k, I) with k + 1 = i. 

Lemma 6.8 F[i) is a non- decreasing concave function. 

Proof. Let A(i) = F{i + 1) — F{i). It suffices to show that the sequence of increments A(i) 
is non-negative and non-increasing. For any i, we define a pair {ki,li) = (|_^J ; I* can be 

verified that all pairs {ki,li) as well as {ki + are balanced. Furthermore, ki + li = i (and 
consequently /cj + 1 + /j = i + 1), so that f{ki + — f{ki,li) = Also, both {/c,,} and {/j} 

are non-decreasing sequences. The decreasing marginal values of the sub modular function / imply 
that A(i + 1) = f{ki+i + l,k+i)-f{ki+i,li+i) < f{ki + l,li)-f{ki,li) = A(z), showing that A(i)'s 
are non-increasing. The monotonicity of / implies that they are also non-negative. □ 

We next define two sequences of pairs, {kf, if) and (/c^, l^), ranging from i = to i = n, which 
we call the K-biased sequence (or walk) and the L-biased sequence, respectively (see Figure [I]). 
The properties of these two sequences will be used in the remainder of the proof. The definitions 
are inductive, with both sequences starting at (0,0). 

(,K ,K . ^ f(^f + 1, If) if kf<lf.K 

[ikf, lf + 1) if kf>lf.f 
,L ,L , _ iikt + h Zf) if </f -f 



if > • f 



Let us call the change from {ki,li) to (fci+i, h+i) in either of the two sequences a K-step if the first 
component of the pair increases by one, and an L-step if the second component increases. The only 
difference between the two sequences is that when equality k = I ■ K/L holds, we take a ET-step in 
the case of the X-biased sequence, and an L-step in the case of the L-biased sequence. For both 
sequences it holds that kf + if = kf' + if = i, kf and kf range between and K, and if and 
range between and L. 

Lemma 6.9 All pairs in the K-biased and L-biased sequences are balanced. 

Proof. The proof is by induction, and it is the same for both sequences, so we denote either 
sequence by {ki, li). The first pair (0, 0) is balanced. Now we assume that the pair {ki, U) is balanced, 
and would like to show that the pair (fci+i, ^i+i) is also balanced. Suppose (fcj+i, h+i) = {hi + 1, /»). 
Then it must be that ki <li ■ Then 

K K 
k- — -2<ki< ki + l < li. — + l. 

If (A;j+i,/j+i) = {ki,li + 1), then it must be that ki > li ■ Then 

K K K K 

{k + 1)- < k- — < h < + 2 < {li + l)- — + 2, 

with the leftmost inequality following because K/L < 1. □ 
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Lemma 6.10 In the K-biased sequence, every K-step is followed by at most j"-^] L-steps. In the 
L-biased sequence, every L-step is followed by at most one K-step. 

Proof. Suppose that the iT-biased sequence, after some pomt {k,l), takes one K-step followed 
by 1"-^] L-steps, reaching the point {k -\-l,l + |"-^] )• Since the step after {k, I) is a K-step, it must 
be that k < IK/L. So 

^ L' 
K 



1 + 



— > I - — + 1 > k+1, 



which means that the next step in the iC-biased sequence will be a i^-step. 

Similarly, for the L-biased walk, suppose that from some point {k,l), the sequence takes an 
L-step, followed by a K-step, reaching the point {k + \,l + 1). Then k > IK/L implies that 

K K K K 

(/ + !)• — = 1- — + — < 1- — + 1 < k + l, 

L Li Lj Li 

and thus the next step is an L-step. □ 

Proof of Lemma 16.41 To lower-bound the value of /(/c, 0), we consider the iiT-biased walk from 
(0,0) to a point (/c,/') which is the last point before the K-step to {k + 1,-). We let f{k,0) = 
-^(0) + Sj=i '^(j)' where 5{j) = f{j, 0) — f{j — 1,0). For each K-step in the K-hiased walk, where 
kt^ =j-l and kf = j, let A^(j) = fikf,lf) - f{kf_„lf_,) = f{j,lf) - f{j - l,lf_,). By 
submodularity of / it follows that A^{j) < d{j). 

We claim that Yl^^^i ^^U) > Ifik, I') - ^(0)]/(l + \^] )• In other words, at least 1/(1 + \^] ) 
fraction of the increase in F{-), as we proceed in the -fC-biased walk, is due to the i^T-steps. This 
follows from several observations. First, the if-biased walk starts with a K-step. Second, by 
Lemma 16. 10^ each i^-step is followed by no more than [-^J L-steps. And third, A^(j) is a 
decreasing sequence (by concavity of F) . 

Further, by concavity of F, we have that f{k,l') > ^^F{n). By definition of I', we have 
/' > kL/K. Also, 1 + 1"-^] < 2{L/K + 1). Putting everything together, we have 

/(fe,o) = F(o) + x;5(j) > f(o) + x:a^(,) > F(o) + ^^Y? r"i.T ^ rrm 

j=i j=i ■'■+1x1 ■'■+1x1 

^ k + l' Fin) ^ k{L/K+l) F{n) k ^. , 

^ ^ = P ( Tl ) 

n 2{L/K+l) - n 2{L/K+l) 2n ^ ' 

To bound f{0,l), we consider the L-biased walk from (0,0) to {k',l) for some k' . Because 
of concavity of F, the L-steps in the walk account for at least half the increase in /, yielding 
/(O, I) > y{k', I). Also, f{k', I) > ^F{n) > {F{n). So we get that /(O, > ^F{n). □ 
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